EP0625853 



Publication Title: 
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Abstract: 

A moving image encoder comprising: a contour extractor for dividing a local 
decoded image into a plurality of segments, and extracting contour information 
therein for each of an n number of frames, n being a natural number, in which 
encoding has been completed; a motion parameter extractor for extracting a set 
of motion parameters based on the contour information for each of an n number 
of frames in which encoding has been completed; a motion compensator for 
forming a prediction image based on the local decoded image, the contour 
information, and the set of motion parameters for each of an n number of frames 
in which encoding has been completed; an encoder for forming encoded 
information by means of quantizing a differential signal of the prediction image 
with a present frame; a local decoder for adding the prediction image to a signal 
formed by inverse quantization of the encoded information, forming a local 
decoded image, and storing this local decoded image into frame memory; and a 
transmitting unit for transmitting the encoded information and the set of motion 
parameters for each of an n number of frames in which encoding has been 
completed. 
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(§) Moving image encoder and decoder. 

@ A moving image encoder comprising : a con- 
tour extractor for dividing a local decoded im- 
age into a plurality of segments, and extracting 
contour information therein for each of an n 
number of frames, n being a natural number, in 
which encoding has been completed ; a motion 
parameter extractor for extracting a set of mo- 
tion parameters based on the contour infor- 
mation for each of an n number of frames in 
which encoding has been completed; a motion 
compensator for forming a prediction image 
based on the local decoded image, the contour 
information, and the set of motion parameters 
for each of an n number of frames in which 
encoding has been completed ; an encoder for 
forming encoded information by means of 
quantizing a differential signal of the prediction 
image with a present frame ; a local decoder for 
adding the prediction image to a signal formed 
by inverse quantization of the encoded infor- 
mation, forming a local decoded image, and 
storing this local decoded image into frame 
memory; and a transmitting unit for transmit- 
ting the encoded information and the set of 
motion parameters for each of an n number of 
frames in which encoding has been completed. 
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Background of the Invention 

The present invention relates to a moving image 
encoder and decoder for performing encoding and 
transmission of a moving image in a more efficient 
manner. 

Relevant Art 

A regional based coding method is known for div- 
iding an Image into regions by means of conducting 
an edge detection process on the image or a process 
in which portions of uniform motion are integrated; 
forming motion parameters which indicate how each 
of the aforementioned regions is modified with re- 
spect to the original image; and transmitting the cod- 
ed motion parameters ("Object-Oriented Analysis- 
Synthesis Coding of Moving Images", H. G. Hus- 
mann, et al. f pp. 117-138, Signal Processing, Elsvier 
Science Publishers B.V., 1989). According to conven- 
tional methods, in addition to obtaining motion infor- 
mation in the aforementioned manner, the contour in- 
formation of each region is coded at each time period 
and then transmitted to the decoder. Fig. 8 Is a block 
diagram showing a construction of a conventional 
moving image encoder for enooding and transmitting 
motion parameters and contour information. As 
shown in Fig. 8, this moving image encoder compris- 
es a differentiator 1, discrete cosine transform 2, 
quantizer 3, inverse quantizer 4, inverse discrete co- 
sine transform 5, adder 6, frame memory 7, motion 
compensation 8, contour extractor 9, and motion 
parameter extractor 10. 

In the aforementioned structure, the input image 
l„ to be coded and transmitted is initially inputted into 
differentiator 1. The difference between input image 
In and prediction image P n Is calculated by means of 
this differentiator 1 , and a differential image An is sub- 
sequently outputted. Prediction image P n will be men- 
tioned hereafter. Subsequently, with regard to this 
differential image A„, direct transforms such as dis- 
crete cosine transform is conducted by means of dis- 
crete cosine transform 2 and the resultant transform 
coefficient C„ Is then outputted. This transform coef- 
ficient Cn is quantized by means of quantizer 3 and 
then sent to the receiving set as coded information D„. 
This coded information D n is sent to both the receiving 
set, and Inverse quantizer 4 where it is quantized. In- 
verse discrete cosine transform is conducted on this 
inverse quantized information by means of an inverse 
discrete cosine transform 5, and quantization differ- 
ential signal QA„ is then outputted. Between a quan- 
tized differential image QA n and differential Image A„, 
a difference equivalent to the quantized error gener- 
ated during the quantization of quantizer 3 exists. The 
quantized differential image OA* is then added to the 
prediction image P n by means of adder 6. The result 
of this addition, i.e., the sum of quantized differential 



image QAn and prediction image P n corresponding to 
the coded image D n sent to the receiver - the image 
information which is actually sent to the receiver 
(hereinafter referred to as local decoded image") - is 
5 then obtained. The image information obtained from 
this adder 6 is then recorded in frame memory 7 as 
a local decoded image. 

On the other hand, the moving vector (V K , V y ) is 
detected by means of a moving vector detector using 

10 a detection method such as a block-matching method 
or the like, and this moving vector and an input image 
l n are then inputted into a contour extractor 9. By 
means of this contour extractor 9, portions possess- 
ing similar motions in input image l n are extracted to- 
ts gether based on the edge information incorporated 
into this input image l n and the motion vectors (V x , V y ). 
The input image l n is then divided (segmented) into a 
plurality of segments each formed from portions pos- 
sessing similar motions. In this manner, the contour 

20 information S n indicating the contours of each seg- 
ment is extracted by means of contour extractor 9; 
this contour information S n is transmitted to the re- 
ceiving set and also sent to motion compensation 8 
and motion parameter extractor 1 0. 

25 In motion parameter extractor 10, with regard to 
the motion vector of a segment within contour S n , the 
optimum affine transform parameters a f signify- 
ing the mean square root error is calculated for each 
segment, and then transmitted to both the receiving 

30 end and motion compensation 8. Motion compensa- 
tion 8 reads out the local decoded image lc„. 1 corre- 
sponding to the frame in which transmission has been 
completed; activates the affine transform parameters 
corresponding to this aforementioned segment as a 

35 set of motion parameters with regard to each pixel 
within each segment designated by the contour infor- 
mation S„ in this local decoded image iCn. 1; and then 
calculates the prediction pixel value of the pixels with- 
in each segment 

40 The aforementioned motion vector is calculated 
by searching for the affine transform A, which mini- 
mizes the evaluation function J, obtained as shown 
below. 

Ji ■ gH(N,ij) - lc(N-1,A,PJD] 

45 wherein, 

g; evaluation function (L 1t L 2 , etc.); 
lc(N- 1 , A,p,fl)): pixel value of A,[i j] of the coded 
image at time point N-1; 

l(N,i,j): pixel value of coordinate (i,j) belonging 
50 to region R of an input image at time point N; 

A, represents the transform from I to Ic of re- 
gion Rc. 

It is possible to evaluate the same procedure us- 
ing the inverse transform of A: as shown below. Sim- 
55 ilarly, the motion vector is calculated by searching for 
the affine transform A J™ which minimizes the evalua- 
tion function J t ,nv . 

Ji ,nv = g[l(N,A 1 lm Ti,J] - lc(N-Uj)] 
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wherein, 

g: evaluation function; 

IctN-Uj): pixel value of coordinate (ij) of the 
coded Image at time point N-1; 

l(N f A 1 lw Pj]): pixel value of coordinate A^J] 5 
belonging to region R of an Input image at time point 
N; 

A, 1 ™ represents the aff ine transform from I to 
Ic of region Rc. 

Fig. 10 is a general diagram showing a regional 10 
prediction according to a conventional image trans- 
mission method. According to this method, the region 
to be predicted in the anticipated image (N frame) is 
initially determined, and the contour Information of 
the portions exhibiting similar motions is then con- 15 
cretely determined. Subsequently, the region of the 
previous image (N-1 frame) to be used in order to pre- 
dict the aforementioned region (to be predicted) is de- 
termined. 

Fig. 9 is a block diagram showing a construction 20 
of a moving image decoder which is used together 
with a conventional moving image encoder. Decoder 
90 comprises inverse quantizer 91, inverse discrete 
cosine transform 92, adder 93, frame memory 94, 
contour reproducing portion 95, and motion compen- 25 
sation 96. In the conventional moving image trans- 
mission process, coded information D n is added to the 
motion parameters a, ... t f, and the contour informa- 
tion S n is then transmitted to the receiving set The 
reason for this type of procedure is explained in the 30 
following. 

Initially, as shown in Fig. 9, in the receiving set, 
the local decoded image is restored by means of per- 
forming inverse quantization and inverse discrete co- 
sine transform of the coded information D n ; the input 35 
image l n at the sending end is then restored by adding 
the prediction image generated at the receiving end 
to the aforementioned local decoded image. This re- 
storation of the prediction image is performed by 

means of activating each motion parameters a f 40 

received from the sending set with regard to the local 
decoded image, as shown in Fig. 10. However, since 
each of the aforementioned sets of motion parame- 
ters is defined for each segment into which various in- 
put images l n are divided, information relating to each 46 
set of motion parameters, as well as the segments in 
which these parameters are supposed to be activated 
is unnecessary in the above restoration of the predic- 
tion image. Conventionally, in the sending set, the 
calculated contour information S„ is coded using so 
chain characters, polygonal approximations, etc., and 
then transmitted to the receiving set. At the receiving 
set, based on this contour information S ni the local de- 
coded image is divided into a plurality of segments, 
and the motion parameters corresponding to the va- 55 
rious segments are activated to obtain the prediction 
image. 

The procedural flow of the moving image encoder 



and moving image decoder In the conventional image 
transmission process is illustrated by the flowchart 
shown in Fig. 1 1 . In the moving image encoder, based 
on both the present image and the previous image, 
the motion parameters, contour Information, and dif- 
ferential information are extracted and transmitted to 
the decoder. In addition, the reconstructed image is 
formed in the decoder based on the aforementioned 
information sent from the encoder. 

Fig. 13 Is a general diagram showing a visual il- 
lustration of the procedural flow based on the conven- 
tional method. At the sending end, upon receipt of the 
image information (300) to be sent (input image In), 
the image is divided into segments by means of an 
edge detection process or the like, and the contour 
data of the segments is then sent to the receiving end 
(301). Subsequently, at the sending end, motion 
parameters are extracted (302) based on the input 
image l n and local decoded image ten. and then sent 
to the receiving end. Furthermore, at the sending end, 
the original activated area of local decoded image Ic^ 
i to activate the motion parameters is calculated 
(303), the motion parameters are activated (304), and 
the prediction Image P„ is formed (305). Lastly, the 
difference between the input image l n and prediction 
image P„ is obtained and sent to the receiving end. 

In the receiving end, the original activated area is 
calculated based on the contour data and motion 
parameters received (307). The motion parameters 
received with respect to the recorded original activat- 
ed area of decoded image l^ are then activated 
(308), and prediction image P„ is formed (309). This 
prediction image P ft is formed based on the same in- 
formation as used in the sending set, thus this predic- 
tion image P n is Identical to the prediction image P n 
obtained in the sending set. The input image In is then 
reproduced by performing inverse quantization of the 
encoded information received and then adding this 
result to the prediction image P n (310). 

However, when processing in this manner, there 
exists a problem in that in the case when contour in- 
formation S n is incorporated into the information sent 
from the sending set to the receiving set, the entire 
amount of Information sent becomes significantly 
large. In addition, the shape of the segments be- 
comes complex, and moreover, in the case when a 
large number of segments exists, the amount of infor- 
mation to be sent further increases, thereby causing 
problems such as the reduction of the transmission 
efficiency. 

Summary of the Invention 

In consideration of the aforementioned, it is an 
objective of the present invention to provide a moving 
image transmission method wherein the amount of in- 
formation to be transmitted can be reduced such that 
a high transmission efficiency can be obtained. In or- 
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der to achieve this objective, the present invention 
provides an image coding method in which the motion 
parameters of each segment are calculated and pre- 
diction Is conducted in between frames. In this meth- 
od, the encoder of the sending end and the decoder 5 
of the receiving end independently determine each 
respective segment with regard to the images of an 
n number of frames (n is a natural number) in which 
encoding has been completed. In order to predict the 
corresponding portion of the present frame from a 10 
segment calculated from the image of the previous 
frame, the encoder calculates the required motion 
parameters, and then sends these motion parame- 
ters with the coded information to the decoder In the 
decoder, prediction between frames is performed us- 15 
ing the segment information independently deter- 
mined with regard to the image of the previous frame 
in addition to the coded information and motion para- 
meters received from the encoder. 

According to the moving image transmission 20 
method of the present invention, since the local de- 
coded image which has undergone decoding is divid- 
ed into segments at both the sending and receiving 
ends according to the same regional segmenting pro- 
cedure, it is not necessary to transmit the contour In- 25 
formation. Consequently, since the information to be 
transmitted comprises only the coded information 
and the motion parameters, the amount of informa- 
tion to be transmitted can be significantly reduced. 
When using a plurality of coded frames in which en- 30 
coding has been completed, prediction in between 
frames is conducted at each segment by referencing 
an M number of frames from among an N number of 
frames in which encoding has been completed (M < 
N, or M = N). 35 

Furthermore, according to an embodiment of the 
present invention, in the case when null portion are 
generated in the present frame which was predicted 
according to the aforementioned method, in other 
words, when the prediction involves a region which 40 
does not exist, or when a region which cannot be pre- 
dicted from the previous frame image is generated, a 
predetermined interpolation procedure is executed. 
Furthermore, according to another embodiment of 
the present invention, a predetermined overlap proc- 45 
ess is executed when overlap exists with regard to 
the present frame predicted by means of the afore- 
mentioned method. 

Hence, according to the embodiments of the 
present invention, by means of executing the afore- 50 
mentioned interpolation and overlap procedures, rap- 
id image reproduction can be accomplished with re- 
gard to an unpredicted region and/or a region in which 
prediction overlap exists. 

55 

Brief Description of the Drawings 

Fig. 1 is a block diagram showing a construction 



of a encoder for conducting a moving Image transmis- 
sion method according to an embodiment of the pres- 
ent invention. 

Fig. 2 is a block diagram showing a construction 
of a decoder for conducting a moving Image transmis- 
sion method according to an embodiment of the pres- 
ent invention. 

Fig. 3 is a general diagram showing image predic- 
tion according to the present invention. 

Fig. 4 Is a general diagram showing an example 
of interpolation. 

Figs. 5(A) and 5(B) are flowcharts showing a 
moving image transmission method according to the 
present invention. 

Fig. 6 is a block diagram showing another con- 
struction of a encoder for conducting a moving image 
transmission method according to the present inven- 
tion. 

Fig. 7 is a block diagram showing another con- 
struction of a decoder for conducting a moving image 
transmission method according to the present inven- 
tion. 

Fig. 8 is a Nock diagram showing a construction 
of a encoder according to a conventional segment 
coding transmission using motion compensation. 

Fig. g is a block diagram showing a construction 
of a decoder according to a conventional segment 
coding transmission using motion compensation. 

Fig 10 is a general diagram showing image pre- 
diction according to a conventional method. 

Figs. 11(A) and 11(B) are flowcharts showing a 
moving image transmission method according to a 
conventional method. 

Fig. 12 is a general diagram showing encoding 
and decoding according to a conventional method. 

Fig. 13 is a general diagram showing encoding 
and decoding according to the present invention. 

Detailed Description of the Preferred Embodiments 

In the following, the preferred embodiments of 
the present invention with reference to the figures. 
Fig. 1 is a block diagram showing a construction of a 
encoder for conducting a moving image transmission 
method according to an embodiment of the present 
invention. Furthermore, in this figure, structures cor- 
responding to those shown in the aforementioned Fig. 
8 are denoted by the same numerals. 

In the apparatus shown in Fig. 8, contour extrac- 
tor g conducts division of an input image l„ Into a plur- 
ality of segments and extracts contour information 
S„. In contrast, contour extractor 9 of the present em- 
bodiment performs regional division of the Image to 
be used in prediction. In other words, this contour ex- 
tractor 9 according to the present embodiment initially 
conducts edge detection of a local decoded image 
lev , of a frame in which transmission has been com- 
pleted and read out from frame memory 7 based on 



4 



7 



EP0 625 853 A2 



8 



a brightness or color differential signal, and then per- 
forms division of this image into segments and ex- 
traction of the contour information S„. 

According to the present invention, based on the 
region of the previous image which has already been 
coded and stored in frame memory, the transform 
process at the time of predicting the portion corre- 
sponding to the prediction image is expressed by 
means of motion parameters. 

The motion vector is calculated as shown below, 
by searching for the aff ine transform A2 which mini- 
mizes the evaluation function J 2 . 

J2 = g[1<N f AdlJD - MN-Uj)] 

wherein, 

g: evaluation function (L,, norm, etc.); 

lc<N-1 pixel value of coordinate (i.j) belong- 
ing to region Rc of the coded image at time point N- 
1; 

KN.AJJ]): pixel value of the Input image at 
time point N; 

A 2 represents a transform from Ic to I of region 

Rc. 

It is possible to evaluate this same procedure us- 
ing the inverse transform of A 2 . At this time, the mo- 
tion vector can be calculated by searching for the af- 
f ine transform A^ which minimizes the evaluation 
function JJ m > as shown below. 

Ja* = gP(NJJ) - lc(N-1,A 2 ^nj])] 
wherein, 

g: evaluation norm (L 1t L 2 , etc.); 
lc(N-1 , AJ™ P.jl): Pixel value of coordinate A 2 |IW 
belonging to region Rc in the coded image at time 
point N-1; 

KN,IJ): pixel value of the input image at time 
point N; 

A 2 tnv represents an aff ine transform from I to 
Ic of region Rc. 

Fig. 3 is a general diagram showing regional pre- 
diction according to the moving image transmission 
method of the present invention. According to the 
present invention, initially, a region is determined in 
the previous Image (N-1 frame). Subsequently, the 
region of the prediction Image (N frame) correspond- 
ing to this aforementioned region is determined. 

In addition, motion parameter extractor 10 shown 
in Fig. 1 is similar to the structure shown in the afore- 
mentioned Fig. 8; this motion parameter extractor 10 
calculates the optimum motion parameters which will 
minimize ths mean square root error with regard to a 
motion vector of a region within contour S n -1. With re- 
gard to the motion parameters, various processes 
can be used; however, a representative process is 
one in which six parameters are used based on the af- 
fine transform. The motion vector is calculated by 
means of performing a block-matching method or the 
like for each pixel; upon calculating the motion vector, 
the six parameters are determined by means of a 
minimum square root approximation method. It is 



possible to describe this motion amount by means of 
the amount of displacement, the amount of rotation, 
and the longitudinal strain. The horizontal • vertical 
motion vector (Vx(x,y), Vy(x,y)) at point (x,y) can be 

5 approximated using aff ine parameters a f in the 

following manner. 

Vx (x,y) *abx + c 
Vy (x.y) d e y + f 
At this point, the affine transform parameters 

10 may be calculated by obtaining the motion vector (Vx, 
Vy) of each pixel. However, it is also possible to obtain 
the motion vector between the two regions wfthout 
calculating Vx and Vy from the relationship between 
the pixel values of the two regions. 

15 In the present embodiment, the case of a two-di- 
mensional affine transform Is described; however, it 
is also possible to use a process in which the three- 
dimensional affine transform is calculated and pro- 
jected onto a two-dimensional plane. Motion oompen- 

20 sation 8 divides local decoded image ICn. , according 
to contour information Sn-1 , in other words, according 
to the contour information used in order to obtain each 
segment in which extraction of the motion parameters 
from local decoded Image lc fv1 is performed. Each 

25 set of motion parameters supplied from the motion 
parameter extractor 1 0 of each segment obtained by 
the aforementioned division is activated to form the 
prediction image P n . 

In the aforementioned, an explanation was pro- 

30 vided with regard to a method for forming the predic- 
tion image using only the previous frame; however, in 
the case when delays are allowable, and/or in the 
case when decoding is performed after accumulation 
of the coded data, in point of time, even a future 

$5 frame can be used In the formation of a prediction im- 
age so long as the coding process has been complet- 
ed. 

Generally, the prediction image is formed using 
a plurality of frames in which encoding has been conv 

40 pleted. When coding is attempted on a frame at time 
point N, the segment of the present frame can be pre- 
dicted using the motion parameters from each of the 
frames at time point N-1 and time point N+1, as long 
as encoding has been completed In these frames. In 

45 addition, at this time, a process can be employed for 
selecting the use of either one of the predictions, or 
the arithmetic mean of both prediction pixels to form 
the prediction image. There are also cases in which 
two or more reference frames are used. Generally in 

so the case when a plurality of frames in which encoding 
has been completed are used, prediction in between 
frames can be conducted by referencing an M num- 
ber of frames from among an N number of frames in 
which encoding has been completed (M < N) at each 

55 segment. 

For the sake of comparison, a conventional meth- 
od for predicting a segment of N frame using a seg- 
ment of N-1 frame will be explained in the following. 
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In the conventional apparatus shown in Fig. 8, the 
segments in which extraction of each of the motion 
parameters is performed are determined by means of 
input image l n . 

As shown in Fig. 1 0, the segments in input image s 
l n are combined such that no overlap exists; however, 
in contrast, the segments of the corresponding local 
decoded image Ic*. , exhibit cases in which there is 
overlap. In Fig. 10, segments 130 and 140 of N-1 
frame correspond to segments 110 and 120 of N-1 w 
haiile; however, overlapping portions exist within seg- 
ment 130 and segment 140. In addition, in local de- 
coded image Icn. 1( there are also cases in which por- 
tions not used in the prediction of input image l n exist, 
and furthermore, there are cases in which segments is 
which cannot be predicted from local decoded image 
lev , exist in an input image l n . According to the con- 
ventional method, the region of the image to be pre- 
dicted is determined in the encoder In order to predict 
this region, coding is carried out by means of motion 20 
parameters using pixels at a predetermined position 
of the previous image. As a result, when the division 
of the region of the image to be predicted is performed 
without overlap, unpredlcted portions are not gener- 
ated (however, as mentioned above, it is still possible 25 
for N-1 frame to lack information required for predic- 
tion). 

In contrast, according to the method of the pres- 
ent invention, as shown in Fig. 3, instead of dividing 
the information of the previous image without excess 30 
or deficiency, there are cases in which a region of the 
present image, as seen from region 31, is predicted 
from two or more regions of the previous image. On 
the other hand, there also exist cases in which a re- 
gion exist 8 without a prediction value according to the $5 
prediction based on the previous image. The former 
case requires calculation of a mean or the like of the 
two overlapping prediction values, while the latter 
case requires an interpolation procedure. With regard 
to region 31 in which the prediction value is obtained 40 
from two or more regions of the previous image, it is 
possible to employ methods which utilize the mean 
value of the prediction values, as well as a process 
which first observes the direction of motion and then 
utilizes the prediction value indicating a motion differ- 45 
ing from the motion vector of the previous time point 

Fig. 4 is a general diagram showing the case in 
which an interpolation procedure is required. In the 
present image 40, three regions 41 , 42, and 43, which 
are predicted from the previous image, exist; how- so 
ever, another region 44 which is surrounded by these 
three regions exists as a portion which is not predict- 
ed from any segments of the previous screen. With 
regard to this portion 44, since the prediction value 
cannot be obtained, It is necessary to predict the im- 55 
age Information using another process, such as an in- 
terpolation process. As this interpolation process, a 
large number of processes can be considered such as 



a process which uses the mean value of the periph- 
eral pixels of the prediction image as the prediction 
value of all pixels of the interpolation area; a process 
which uses the mean value of the pixels of the right 
and left peripheral portions of the interpolation area 
with regard to the horizontal direction; a process 
which uses the mean value of the pixels of the top and 
bottom peripheral portions of the interpolation area 
with regard to the vertical direction; a process which 
examines the motion direction of the right and left por- 
tions of the Interpolation area, focuses on the region 
exhibiting motion in the direction towards the interpo- 
lation area, and utilizes the pixel value at the contact 
point with the interpolation area of this region; a proc- 
ess which focuses on the region exhibiting motion in 
the vertical direction, and utilizes the pixel value at 
the contact point with the interpolation area of this re- 
gion; a process which focuses on the horizontal and 
vertical motions on the periphery of the interpolation 
area and utilizes the pixel value of the region exhib- 
iting motion into the interpolation area; and a process 
for interpolating in pixel units using the motion vector 
which either interpolates or extrapolates surrounding 
motion vectors using pixel units. 

Post processing part 11 inFig. 1 performs correc- 
tion of the prediction image according to the interpo- 
lation calculations. In other words, this post process- 
ing part 11 performs the Interpolation calculation on 
the pixel values of each pixel comprising the predic- 
tion image, and then calculates the pixel value of the 
prediction image. In addition, in accordance with a 
predetermined priority order, the pixel value from the 
interpolation, or any pixel value of the prediction Im- 
age is designated as the pixel value of the prediction 
image. In this manner, the prediction image P n to be 
supplied to the differentiator 1 is calculated. In the 
same manner as in the conventional apparatus, dif- 
ferentiator 1 outputs the difference between the input 
image l n and the prediction image P n . However, when 
the difference between the input image and in the 
prediction image is extremely large, in other words, 
when the norm from using the prediction image is 
larger than the norm without using the prediction im- 
age, a judgment is rendered that use of the prediction 
image Is inappropriate. The post processing part 11 
then sets each pixel value of the prediction image of 
the corresponding region to "0", and differentiator 1 
sends the input image in its original form to the dis- 
crete cosine transform 2. As In the aforementioned 
apparatus shown in Fig. 2, discrete cosine transform 
2 and quantizer 3 transform the differential image An 
into coded information Dn and then transmit it to the 
receiving set The actions of inverse quantizer 4, in- 
verse discrete cosine transform 5, adder 6, and frame 
memory 7, provided as elements of a separate struc- 
ture, are the same as In the apparatus shown in the 
above Fig. 2. 

The decision whether or not to carry out the Inter- 
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poiation procedure is conducted according to the fol- 
lowing procedure. Namely, the portion to undergo the 
interpolation process in the post processing part is a 
portion which cannot be predicted by modification of 
a certain portion of the image of the previous frame. 
In the case when there is no correlation between the 
pixels of the portion to be interpolated and the periph- 
eral pixels, there is no improvement of the prediction 
efficiency even when prediction of the pixels of the 
present frame is performed using the interpolation 
pixels. This relationship can be expressed by the fol- 
lowing formula. 

a = f[l - P(ij)] 
t> = fPOJM 

wherein, 

I (IJ) represents the input image incorporated 
into region R from which the contour was obtained; 

P(i.j) represents the prediction image of region 
R obtained by means of interpolation processing; 

f [•! represents a random norm from among L,, 
L 2 L 

A small norm as expressed by the aforemen- 
tioned formula signifies a high prediction efficiency. 
Consequently, a and b are compared, and when a < 
b, the Interpolation process is performed, while when 
a > b or a = b, it is not necessary to perform the inter- 
potation process. Even if the prediction value is set to 
"0" without performing the interpolation process, a 
prediction error corresponds to the encoding of a pixel 
of a new portion within the same frame. In the case 
when the prediction is completely evaluated, encod- 
ing the new portion within the same frame represents 
the means for obtaining the highest coding efficiency. 
Consequently, even when the interpolation process is 
not performed, a constant result can be obtained for 
the coding efficiency. 

In the aforementioned, a case was presented in 
which motion parameters were used in the formation 
of the prediction image by means of segments, how- 
ever, in addition to these motion parameters, the pre- 
diction image can be formed by means of brightness 
and/or contrast compensation. If the brightnesses of 
the segments are uniformly modified, the prediction 
image can be formed by correcting this value. In ad- 
dition, when the contrast varies, the prediction image 
can be formed by adjusting this contrast The order of 
performing these processes utilizing the aforemen- 
tioned corrections or motion parameters, can be op- 
tionally designated. 

In the present embodiment, the encoding infor- 
mation Dn and motion parameters are sent from the 
sending set to the receiving set, however, the contour 
information S n -1 is not transmitted. In the receiving 
set, the input image is restored as described below 
based on the aforementioned coded Information Dn 
and motion parameters. 

As shown in Fig. 2, in the receiving set, inverse 
quantization is initially performed on the coded infor- 



mation Dn received from the sending set, and the re- 
sult therein subsequently undergoes Inverse discrete 
cosine transform. In this manner, the Inverse trans- 
form of the transform carried out in the sending set to 
s obtain the coded information Dn of the differential im- 
age An is performed in the receiving set In order to re- 
store the image corresponding to the aforementioned 
differential image An. Subsequently, the prediction 
image of the receiving end at this time point Is added 

10 to this differential image An, and the local decoded 
image at the receiving end, which Is identical to the 
local decoded image formed in the sending set, is the 
restored. The local decoded image is then divided into 
a plurality of segments in which motion Is uniform by 

f5 means of the exact same process as performed in the 
sending set, and each of these segments is then 
modified by applying the motion parameters re- 
ceived. With regard to this result, correction of the 
prediction image Is performed In an identical manner 

20 to the process performed In the post processing part 
of the sending set to form the prediction image of the 
receiving end. 

In this manner, in the receiving set a prediction 
Image identical to that formed in the sending set is 

25 created by dividing the local decoded image and ap- 
plying the motion parameters in the same manner as 
in the sending set without having to transmit or re- 
ceive the contour information for each segment from 
the sending end. 

30 Fig. 5 is a flowchart showing the procedures in 
both the encoder and decoder according to the afore- 
mentioned embodiment 

Fig. 12 is a general diagram showing a visual il- 
lustration of the procedural flow based on the present 

35 invention which corresponds to the flow diagram 
based on the conventional method shown in Fig. 13. 
At the sending end, upon receipt of the image infor- 
mation (200) to be sent (Input image IJ, the image is 
divided into segments by means of performing an 

40 edge detection process or the like, and the contour 
data of the segments is then sent to the receiving end 
(201). Subsequently, at the sending end, motion 
parameters are extracted (202) based on the input 
image l n and local decoded image Ic^ and then sent 

45 to the receiving end. Furthermore, at the sending end, 
the target activated area of motion parameters is cal- 
culated (203), the motion parameters are activated 
(204), and the prediction image P n is formed (205). It 
is possible for portions lacking a prediction value, as 

50 well as portions with overlapping prediction values to 
exist thus in these cases, a corrected prediction im- 
age Pn' is formed by means of performing an interpo- 
lation operation and/or overlap processing (207). 
Lastly, the difference between the input image l„ and 

55 corrected prediction image P„. is calculated and sent 
to the receiving end. 

In the receiving end, the contour information is 
extracted in the same manner as in the sending end 
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with respect to the recorded decoded image 1 
(209). With respect to this result, the motion parame- 
ters received are then activated, and prediction image 
P n is obtained (211). This prediction image P ft is 
formed based on the same information as used in the 5 
sending set thus this prediction image P„ is identical 
to the prediction image P n obtained in the sending set. 
The input image t„ is then reproduced by performing 
inverse quantization of the encoded information re- 
ceived and then adding this result with the corrected 10 
prediction image P n . (214). 

Furthermore, the present invention is not limited 
to the aforementioned description contained in the 
embodiments, as the present invention can also be 
executed using various other aspects such as those 15 
mentioned in the following. 

(1) With regard to the extraction of the contour in- 
formation from local decoded image, various 
methods can be employed, as long as execution 

of these methods is possible in a similar manner 20 
in the receiving set 

For example, a method may be employed 
which uses notonly the contour information incor- 
porated into the local decoded image, but also 
the number of sets of motion parameters. In other 25 
words, it is possible to use a process In which if 
the number of sets is K, regional extraction algo- 
rithms are activated to reduce the number of re- 
gions in a restoraWe manner such that at the time 
point when the number of regions reaches K, re- 30 
gional division is completed. In addition, uniform 
portions from among past motion parameters can 
also be combined. 

(2) Extraction of the motion parameters is not just 
restricted to procedures involving block match- 35 
ing-type motion compensation, two-dimensional 
affine transform, or the projection of a three- 
dimensional affine transform onto a two-dimen- 
sional plane, as various transforms may be used, 

as long as transforms such as displacement ro- 40 
tation. contraction and the like can be expressed 
using only a few parameters. 

(3) With regard to the interpolation and extrapo- 
lation performed in the post processing part for 
correcting the prediction image, any appropriate 45 
process may be employed, as long as it can be 
similarly executed by means of the receiving set. 

In the same manner, processing of the over- 
lapping areas of the prediction image may be per- 
formed using any appropriate process, as long as 50 
it can be executed by means of the receiving set, 
such as a selection process which utilizes a sim- 
ple arithmetic mean, a weighted mean possess- 
ing an optional ratio, or the direction of the motion 
vector, or a weighted mean which varies the ratio 65 
by means of the direction of the motion vector. 

(4) Encoding of the differential image is not lim- 
ited to just orthogonal transform, such as discrete 



cosine transform and the like, and quantization, 
as it rs also possible to employ various coding 
methods such as differential prediction coding, 
as shown in Fig. 6 (DPCM), and/or coding proc- 
esses which utilize an analysis filter, as shown in 
Fig. 7, inverse transform, synthesis filter, and the 
like. In the analysis • synthesis filter, a parallel fil- 
ter bank or weWet can also be used. 
As explained above, according to the present in- 
vention, a high transmission efficiency can be ob- 
tained wherein encoded transmission of a segment in 
which motion compensation or brightness • contrast 
correction/adjustment is used without transmission of 
the contour information from the sending set to the re- 
ceiving set Furthermore, problems which can be an- 
ticipated at the time of executing the aforementioned 
method, such as those stemming from the occur- 
rence of areas for which a prediction value cannot be 
obtained, as well as from the existence of a plurality 
of prediction values, can be rapidly processed by 
means of interpolation and overlap processing. 



Claims 

1. A moving image encoder comprising: 

a contour extracting means for dividing a 
local decoded image into a plurality of segments 
and extracting contour information from said a 
plurality of segments therein for each of an n 
number of frames, n being a natural number, in 
which encoding has been completed; 

a motion parameter extracting means for 
extracting a set of motion parameters based on 
said contour information for each of an n number 
of frames in which encoding has been completed; 

a motion compensation means for forming 
a prediction image based on said local decoded 
image, said contour information, and said set of 
motion parameters for each of an n number of 
frames in which encoding has been completed; 

an encoding means for forming encoded 
information by means of quantizing a differential 
signal of said prediction image with a present 
frame; 

a local decoding means for adding said 
prediction image to a signal formed by inverse 
quantization of said encoded information, form- 
ing a local decoded image from said signal, and 
storing said local decoded Image into frame 
memory; 

and a transmission means for transmitting 
said encoded information and said set of motion 
parameters for each of an n number of frames in 
which encoding has been completed. 

2. A moving image encoder as mentioned in claim 
1 wherein, said encoding means is constructed 
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such that a differential signal of a present frame 
and said prediction image is transformed and 
quantized to form encoded information; and said 
local decoding means is constructed such that 
said prediction image is added to a signal formed 
by inverse quantization and inverse transform of 
said encoded information to form a local decoded 
image, which is then stored into frame memory. 

3. A moving image encoder as mentioned in claim 
1 further comprising an interpolation means for 
performing an interpolation operation on said 
prediction image. 

4. A moving image encoder as mentioned in claim 
1 further comprising an overlap processing 
means for processing overlap of said prediction 
image. 

5. A moving image encoder as mentioned in claim 
1 wherein, said motion compensating means is 
constructed such that a brightness or contrast is 
corrected to form a prediction image. 

6. A moving image decoder comprising: 

a receiving means for receiving encoded 
information and motion parameters for each of an 
n number of frames, n being a natural number, in 
which encoding has been completed; 

a contour extracting means for dividing 
each decoded image into a plurality of segments 
and extracting contour information from said a 
plurality of segments therein for each of an n 
number of frames in which encoding has been 
completed; 

a motion compensation means for forming 
a prediction image based on said decoded im- 
age, said contour information, and said motion 
parameters for each of an n number of frames in 
which encoding has been completed; 

a decoding means for adding said predic- 
tion image to a signal formed by inverse quanti- 
zation of said encoded information, forming a de- 
coded image from said signal, and storing said 
decoded image into frame memory, 

7. A moving image decoder as mentioned in claim 
6 wherein, said decoding means is constructed 
such that said prediction image is added to a sig- 
nal formed by inverse quantization and inverse 
transform of said encoded information to form a 
decoded image, which is then stored into frame 
memory. 



9. A moving image decoder as mentioned In claim 
6 further comprising an overlap processing 
means for overlap processing said prediction im- 
age. 

5 

10. A moving Image decoder as mentioned in claim 
6 wherein, said motion compensation means is 
constructed such that a brightness or contrast is 
corrected to form a prediction image. 
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8. A moving image decoder as mentioned in claim 55 
6 further comprising an interpolation means for 
performing an interpolation operation on said 
prediction image. 
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